[FIX_FOR_VLLM_CUSTOM=ecf9d83520eb217401b47d8a5451a27c5231b8c2] Adapt HPU scheduler, ngram proposer and offloading connector tests to upstream API drift - #1556
Merged
iboiko-habana merged 4 commits intoJun 22, 2026
Conversation
…d throttle_prefills in HPUAsyncScheduler.schedule Root cause: vLLM PR #44558 added a throttle_prefills positional arg to Scheduler.schedule(); EngineCore calls it positionally but the HPU override took only self. Upstream: vllm-project/vllm#44558 Fix: accept throttle_prefills (default False) and forward it to super().schedule(). Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
…um_speculative_tokens to NgramProposer.propose Root cause: vLLM PR #32374 (Dynamic SD) added a leading num_speculative_tokens positional arg to NgramProposer.propose(). Upstream: vllm-project/vllm#32374 Fix: Prepend self.speculative_config.num_speculative_tokens in propose_ngram_draft_token_ids. Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
…OffloadingConnector stats tests with upstream flat-metrics API
Root cause: vLLM PR #35669 rewrote OffloadingConnectorStats to a self-describing {types, data} payload with flat metric names, dropping the per-direction CPU_to_GPU/GPU_to_CPU list shape the tests still asserted.
Upstream: vllm-project/vllm#35669
Fix: Rewrite test_metrics.py to exercise increase_counter/observe_histogram/aggregate/reduce/reset against the new self-describing stats contract.
Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
pawel-olejniczak
requested review from
PatrykWo,
adobrzyn,
afierka-intel,
iboiko-habana,
jbyczkow,
mgawarkiewicz-intel,
michalkuligowski and
xuechendi
as code owners
June 19, 2026 12:23
pawel-olejniczak
temporarily deployed
to
pre-merge-approval
June 19, 2026 12:23 — with
GitHub Actions
Inactive
…on-finish Root cause: upstream vLLM commit f428718ffe (#45823, "Defer on_request_finished until in-flight transfers drain") changed the OffloadingConnectorScheduler so a finishing request with in-flight store jobs no longer flushes those stores immediately; finalization is deferred until the transfers drain, and flush now fires only on preemption or block reuse. test_concurrent_lookups_of_the_same_prefix and test_abort_loading_requests still asserted flush-on-finish, so they failed once the target vLLM SHA picked up #45823. Upstream's own equivalents assert no flush in these scenarios. Drop the stale expected_flushed_gpu_block_indexes assertions; test_request_preemption keeps its flush-on-preemption assertion, which upstream still honors. Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
pawel-olejniczak
temporarily deployed
to
pre-merge-approval
June 19, 2026 21:28 — with
GitHub Actions
Inactive
iboiko-habana
approved these changes
Jun 22, 2026
Contributor
✅ CI PassedAll checks passed successfully against the following vllm commit: |
slokesha
pushed a commit
to slokesha/vllm-gaudi
that referenced
this pull request
Jul 6, 2026
…HPU scheduler, ngram proposer and offloading connector tests to upstream API drift (vllm-project#1556) ## Bug 1: Forward throttle_prefills in HPUAsyncScheduler.schedule - **State machine id**: hpu_async_scheduler_schedule_positional_arg - **Commit**: 957ba4d ### Root cause vLLM PR #44558 added a throttle_prefills positional arg to Scheduler.schedule(); EngineCore calls it positionally but the HPU override only accepted self. ### Upstream PR vllm-project/vllm#44558 ### Fix Accept throttle_prefills (default False) on the HPUAsyncScheduler.schedule override and forward it to super().schedule(). ## Bug 2: Pass num_speculative_tokens to NgramProposer.propose - **State machine id**: ngram_proposer_propose_missing_positional_arg - **Commit**: 82155ea ### Root cause vLLM PR #32374 (Dynamic SD) added a leading num_speculative_tokens positional arg to NgramProposer.propose(). ### Upstream PR vllm-project/vllm#32374 ### Fix Prepend self.speculative_config.num_speculative_tokens in propose_ngram_draft_token_ids to match the new upstream signature. ## Bug 3: Align OffloadingConnector stats tests with upstream flat-metrics API - **State machine id**: offloading_connector_cpu_to_gpu_metrics_missing - **Commit**: c1eb9e3 ### Root cause vLLM PR #35669 rewrote OffloadingConnectorStats to a self-describing {types, data} flat-metric payload, dropping the per-direction CPU_to_GPU/GPU_to_CPU list shape the tests still asserted. ### Upstream PR vllm-project/vllm#35669 ### Fix Rewrite test_metrics.py to exercise increase_counter/observe_histogram/aggregate/reduce/reset against the new self-describing stats contract. ## Bug 4: Align OffloadingConnector scheduler flush assertions with upstream defer-on-finish - **State machine id**: offloading_connector_flush_on_finish_deferred - **Commit**: 575a178 ### Root cause vLLM commit f428718ffe (PR #45823, "Defer on_request_finished until in-flight transfers drain") changed OffloadingConnectorScheduler: a finishing request with in-flight store jobs no longer flushes those stores immediately — finalization is deferred until transfers drain, and flush now fires only on preemption or block reuse. test_concurrent_lookups_of_the_same_prefix and test_abort_loading_requests still asserted flush-on-finish, so they failed once the target vLLM SHA picked up #45823. ### Upstream PR vllm-project/vllm#45823 ### Fix Drop the stale expected_flushed_gpu_block_indexes assertions in the two affected tests (matching upstream's own equivalents, which assert no flush in these scenarios). test_request_preemption keeps its flush-on-preemption assertion, which upstream still honors. --------- Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com> Signed-off-by: slokesha <slokeshappa@habana.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug 1: Forward throttle_prefills in HPUAsyncScheduler.schedule
Root cause
vLLM PR #44558 added a throttle_prefills positional arg to Scheduler.schedule(); EngineCore calls it positionally but the HPU override only accepted self.
Upstream PR
vllm-project/vllm#44558
Fix
Accept throttle_prefills (default False) on the HPUAsyncScheduler.schedule override and forward it to super().schedule().
Bug 2: Pass num_speculative_tokens to NgramProposer.propose
Root cause
vLLM PR #32374 (Dynamic SD) added a leading num_speculative_tokens positional arg to NgramProposer.propose().
Upstream PR
vllm-project/vllm#32374
Fix
Prepend self.speculative_config.num_speculative_tokens in propose_ngram_draft_token_ids to match the new upstream signature.
Bug 3: Align OffloadingConnector stats tests with upstream flat-metrics API
Root cause
vLLM PR #35669 rewrote OffloadingConnectorStats to a self-describing {types, data} flat-metric payload, dropping the per-direction CPU_to_GPU/GPU_to_CPU list shape the tests still asserted.
Upstream PR
vllm-project/vllm#35669
Fix
Rewrite test_metrics.py to exercise increase_counter/observe_histogram/aggregate/reduce/reset against the new self-describing stats contract.
Bug 4: Align OffloadingConnector scheduler flush assertions with upstream defer-on-finish
Root cause
vLLM commit f428718ffe (PR #45823, "Defer on_request_finished until in-flight
transfers drain") changed OffloadingConnectorScheduler: a finishing request with
in-flight store jobs no longer flushes those stores immediately — finalization
is deferred until transfers drain, and flush now fires only on preemption or
block reuse. test_concurrent_lookups_of_the_same_prefix and
test_abort_loading_requests still asserted flush-on-finish, so they failed once
the target vLLM SHA picked up #45823.
Upstream PR
vllm-project/vllm#45823
Fix
Drop the stale expected_flushed_gpu_block_indexes assertions in the two affected
tests (matching upstream's own equivalents, which assert no flush in these
scenarios). test_request_preemption keeps its flush-on-preemption assertion,
which upstream still honors.